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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 07 June 2006 . 
2a)^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Response to Amendment and Arguments 

1. Applicant's amendment filed June 07, 2006, has been entered and made of record. 

2. The rejection of claims 14-24 under 35 U.S.C. 101 is withdrawn. 

3. The objection of claim 25 is withdrawn. 

4. The Specification's Amendment filed 06/07/2006 is accepted. 

5. Applicant's arguments with regard to claims 1-28 have been fully considered, but are not 
considered persuasive because of the following reasons: 

Regarding claim 1, the Applicant argues (pages 1 1-13 of the Remarks) that Jain 
Reference does not teach the representing important semantic cues in the human perception of 
images because Jain appears unrelated to deriving a set of perceptual semantic categories and 
instead appears related to allowing a user to specify a properties or attribute of an image wherein 
attributes are primitives. The Examiner respectfully disagrees. As indicated by the Applicant: " 
This method is also based on the set of . . . These include, but are not limited to . people, 
landscapes, waterscapes . . .flower, animal, etc. (emphasis added). First, the Examiner has the 
right not to include the reading of specification into the claim's language as shown in section 
2111, Claim Interpretation; Broadest Reasonable Interpretation as follow: "The court explained 
that "reading a claim in light of the specification, to thereby interpret limitations explicitly 
recited in the claim, is a quite different thing from reading limitations of the specification into a 
claim,' to thereby narrow the scope of the claim by implicitly adding disclosed limitations which 
have no express basis in the claim." The court found that applicant was advocating the latter, i.e., 
the impermissible importation of subject matter from the specification into the claim.). See also 
In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 1027-28 (Fed. Cir. 1997) (The court 
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held that the PTO is not required, in the course of prosecution, to interpret claims in applications 
in the same manner as a court would interpret claims in an infringement suit. Rather, the "PTO 
applies to verbiage of the proposed claims the broadest reasonable meaning of the words in their 
ordinary usage as they would be understood by one of ordinary skill in the art, taking into 
account whatever enlightenment by way of definitions or otherwise that may be afforded by the 
written description contained in applicant's specification.")". Also, since semantic 
cues/categories as defined by the Applicant (page 1 1 of the Remarks) that it can be people, 
landscapes, waterscapes, people, objects, flower. . .etc and are not limited to the listed examples 
specified by the Applicant. Therefore, visual attributes to human/user such as color, texture, 
shape, pattern, object (face) (column 4, lines 1-45) are extensive attributes of semantic 
cues/categories in the human perception of images because human can clearly differentiate the 
color, texture, shape, pattern or objects (column 4, lines 20-38). 

Regarding claim 2, the Applicant argues (top of page 14) that there is no disclosure in 
Jain that teaches "the perceptual features and their combinations are derived through subjective 
experiments performed with human observers". The Examiner respectfully disagrees. Again, as 
discussed the claim 1 's argument, the claim's language are broadly claimed and that the 
Examiner can give broadest reasonable interpretation without reading the specification into the 
claim. Furthermore, the specification also gives a limitless definition regarding perceptual 
features/semantic cues/categories (as discussed in claim 1). Therefore, Jain clearly teaches the 
perceptual features (such as color, texture, shape, pattern, and object that are visual to 
human/user) and their combinations (column 4, lines 1-45) are derived through subjective 
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experiments (the process of obtaining similar properties) performed with human observers 
(column 4, lines 1-45). 

For arguments of other claims, please refer back to the explanations above. 

Thus, the rejections of all of the claims are maintained. 

Claim Rejections - 35 USC §102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

7. Claims 1-28 are rejected under 35 U.S.C. 102(b) as being anticipated by Jain et al. U.S. 
Patent No. 5,915,250. 

Regarding claim 1, Jain teaches a computer implemented method (FIG. 1 A) for determining the 
semantic meaning of mages, comprising: 

deriving a set of perceptual semantic categories (defining and register custom primitives) 
(abstract) for representing important semantic cues (object's attributes) (abstract) (color, 
texture, shape, pattern, object) (face) (column 4, lines 1-45) in the human perception of images 
(column 4, lines 20-25), where each semantic category is modeled through a combination of 
perceptual features that define the semantics of that category and that discriminate that category 
from other categories (visual information retrieval system provides modular and framework to 
define and classify category of object's attributes) (abstract and column 3, lines 60 to column 4, 
lines 1-10); and 
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for each semantic category, forming a set of the perceptual features as a complete feature 
set CFS (feature vector to store each kind of primitives) (column 5, lines 5-20). 

Regarding claim 2, Jain teaches a method wherein the perceptual features and their 
combinations are derived through subjective experiments performed with human 
observers (column 4, lines 20-31). 

For claim 3, Jain further teaches a method further comprising extracting perceptual 
features from an input image and applying a perceptually-based metric (a similarity scoring 
system) to determine the semantic category for that image (abstract). 

Referring to claim 4, Jain discloses a method comprising processing the input image to 
compute the CFS (FIG. 1 A, element 1 12); comparing the input image to each semantic category 
through the perceptually-based metric that computes a similarity measure between the features 
used to describe the semantic category and the corresponding features extracted (abstract) from 
the input image (rank similarity result) (column 12, lines 50-67); and assigning the input image 
to the semantic category that corresponds to a highest value of the similarity measure (column 
12, lines 65 to column 13, line 3). 

For claims 5, Jain also discloses a method further comprising computing features from 
the CFS for images in an image database (FIG. 1 A); and generating a distance measure for 
characterizing a relationship of a selected image to another image in the image database by 
applying a perceptually-based similarity metric (column 8, lines 10-27). 

For claim 6, Jain shows a method where values of the similarity metric computed for 
images in the image database are subsequently used to search for similar images 
in the image database (column 9, lines 50-64). 
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Regarding claim 7, Jain shows a method where values of the similarity metric computed 
for images in the image database are subsequently used to organize images in the 
image database (classifying images base on similarity scoring system) (FIG. 5B, elements 280 
and 284). 

For claim 8, Jain teaches a method where values of the similarity metric computed for 
images in the image database are subsequently used to display images in the image database in 
an organized manner (FIG. 4; FIG. 5B, element 294; and column 11, lines 23-39). 

Also to claim 9, Jain further teaches a method further comprising defining a subset of 
features for the selected image or for an image retrieved from the image database, and using the 
subset of features to refine a search through the image database (searching with specific 
parameters) (column 4, lines 29-45). 

Referring to claim 10, Jain shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network (FIG. IB and column 
9, lines 10-25). 

Also to claim 11, Jain further shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the step of 
characterizing the relationship of the selected image to another image in the image database by 
applying the perceptually-based similarity metric is accomplished to retrieve an image from the 
remote image database (FIG. IB and column 9, lines 10-50). 

As for claim 12, Jain also teaches a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the 
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step of characterizing the relationship of the selected image to another image in the image 
database by applying the perceptually-based similarity metric is accomplished in conjunction 
with a text-based search algorithm to retrieve a multi-media object from the remote location 
(direction communication between databases) (column 9, lines 10-50). 

Referring to claim 13, Jain discloses a method wherein to assign a particular semantic 
category to an image all of a set of Required Features must be present in the image, and at least 
one of a set of Frequently Occurring Features must be present in the image (fixed/universal or 
default primitives) (column 8, line 60 and column 16, lines 16-25). 

For claim 14, please refer back to claims 1-3 for further teachings and explanations. In 
addition, Jain teaches a data processing system comprising a data processor, a graphical user 
interface and a memory to processes the aforementioned limitations (FIG. 1 A and column 9, 
lines 5-50). 

For claims 15-24, please refer back to claims 4-13 respectively for further teachings and 
explanations. 

For claim 25, please refer back to claims 1-4 for further teachings and explanations. In 
addition, Jain teaches a computer program (column 6, lines 15-25) to processes aforementioned 
limitation and semantic categories being modeled using multidimensional scaling and 
hierarchical clustering techniques (different ways of querying and classifying images) (FIG. 1 A, 
elements 106 and 108). 

For claims 26 - 27, Jain teaches a computer program where as a result of comparing the 
input image to images stored in the image database one or more most similar 
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images are identified in the image database (column 12, lines 50-67 and (column 12, lines 65 to 
column 13, line 3) and display it (FIG. IB, element 148). 

For claim 28, please refer back to claim 14 for further teachings and explanations. 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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Contact Information 



9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Q. Le whose telephone number is 571-272-7424. The 
examiner can normally be reached on 8:30 A.M - 5:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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